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iMi afSditloii to die Qucs^n Baalc vorleftr 
tea Al#o Jpoftjlol^od a lew fOMereh Atu4i«e« Tlie ^ 

tiieie repeercheA are teing embodied iam pefiei of moticHira^M* 
Aa viPoalUed at preaeot the fdUowiag topic* are beli^ coveredt 

4 

1« Gradiog*. 

ly Qixeitioo Baaklag for U&iveriltlea« 

3« Xaterikal AAieeameBt. 

4. Practical Examinationsi 

5* Teat and Item Analyiia. 

Xbe Monograph on Grading was released last yA^r and 
has been reprinted in its third revised edition just n<^* '^l^e 
Monograph on Question Banking for Universities is presented 
herewith. In a sense* this mondgraph acts as a curtain raiser 
to the series on Question Banking which is being publieked 
separately. This monograph sets oiit the tb«^ry of ^estion 
banking* its uses in university examinations*, levels e^d forms 
of question banking* storing of quotations retrieving them* Pre* 
vslidating them* post-validating them and a whole h^at of 
related issues* As workshops and seminars are bel^S organised 
and will continue to be organised in colleges and univaraities 
all Dver the coosAry *' this monograph wo^m be found particularly 
useful* indeed indispensable* 

Since there still prevails some confusion with regard to 
the objectives behind question banking* it seems ne^^**^fy h> 
repeat that queation banking is by no means an attei^P^ short 
circuit the process of learning. On the contrary* it 1* ^ 
pedagogic device to regulate and quicken die proces* of learning. 
Indeed to participate in the task of setting questions la itself 
sn educative process. Xhis was^e experience of bund 
teachers who have participated uMhe more dian 20 
organised by the Association in different' parte of tb^ 

To be able to ask the right kind of questions require* a 

diorou^ underatanding of the subject matters That i* 

the teachera found that partici^tion in theae workshop* Itolped 



to deopoa their underetandlzig of the eubject« Since the 
educational proceaa ia the aame» whether for ^e teacbere or 
the taught, there ia every^ reaaun to believe that &e atudenta 
alao would profit from it aa much a a the teachera* 

The Monograph on Internal Aaaeaament ia alao ready and 
will be available witiiin the next few weeka. The other 
monographs are under preparation and are expected to be ready 
within the next few months • 

We are confident that each one of these monograph! would 
be found uaeful by the community of teachera and atudenta. 


New Delhi, Amrik Singh 

January 8, 1977. SecreUry 

Aafiocistion of Indian XJniveraities. 





Many educationally developed countries have been for 
sometime carrying out, et all levels of education^ a ay sterna tic 
collection of a bank or pool of items and questions for use. In 
India the NC£RT started developing pools of questions/items for 
school examinations. The Association of Indian UniversitieB 
( then Inter-University 6oard) during a Seminar on Examinations 
in Higher Educatiofn^' conducted in January 1971 recommended 
question bankin^f as a major plank of examination reform. Jn 
' Examination Reform- A Plan of Action' the UGC also suggested 
question banking as part of examination leform for universities. 

The Association of Indian Universities accordingly under¬ 
took Question Banking as a development project and has so far 
successfully built up initial question banks in 10 subjects i. e. 
Mathematics, Physics, Chemistry, Botany, Zoology^ History, 
Geography, Psychology, Economics and Commerce at the 
first degree level. Neatly 60,000 questions/items in these 
subjects are available in specially designed cards in Cardex 
trays. It is also proposed to print Question Bank Books in these 
subjects making available a set of pr©validated qMe 9 t 2 on 3 /items 
of all types in all the ar^as of these subjects. Three iriSin 
sources from which questions/items were collected included: 

a) from specially conducted Question Banking Workshops 
in various universities/colleges for periods ranging 
from 3 days to 10 days where teachers were inducted 
into the science of question making. Nearly 30 to 60 
teachers in every subject were involved. Questions/ 
items made during these workshops account for 30 to 
40% of the total. 

b) from teachers and item writera from all over the 
country. Care was also taken to select around 20 
teachers/ subject who were trained in evaliiation 
through workshops conducted in the past. Nearly 40% 
of total questionsVitems in any bank is com.ppsed of this. 



c) from pa#t oxaminatioii papers of tmivortitiei* Most of 
the questions/items were repeated stereotyped and vaguely 
worded and in effect did not satisfy the criteria for 
quality* Hence, from this soorcei it was not possible 
to have more than 15 to Z0% of total questioas/items in 
each subject. 

These questions/items were prevalidated with a checklist 
of criteria for good quality questions/items and provided with 
basic technical values like a) coding ( Bibliograi^ic); b) subject/ 
topic; c) type of question; d) objective tested; e) time for 
answering and f) marks allotted. 

Thus a bank of items/questions of known technical values 
has been built up for future use. It might, in practice, prove 
of value to arrange for item construction to be a more or less 
continuous process. The construction of written papers can 
then become a matter of judging the suitability of items of known 
technical values from a bank of items. Items can be weeded out 
ae ' out of date’ over a period of time. These banks would 
have to be large to be of ' value' . In fact these banks 
would be not just material for terminal evaluation but also 
for continuous assessment and individualised examining, 

A Monograph detailing fundamental principles, outcomes 
and uses of Question Bank to enable universities to launch 
Upon building initial question banks in various subjects is 
being brought out. Monographs are usually intended: 

a) to share the ideas with other researchers concerned 
to generate more thinking in this area; 

b) to acquaint educational workers such as educational 
administrators^ curriculum planners and teacher 

educators with the tentative Endings of the explorative 
study; 

c) to obtain, if possible, their commrnts and criticisms 
for the beneEt of Ihe project. 

It is.hoped that educational administrators, teachers and 
others connected with university examinaEone will find this 
Monograph useful. Constructive suggestions in regard to its 
improvement for future editions will be moat welcome . 


New Delhi, 
January 8 ,1977 


V • Ketars jea 

Project OEicer ( £xama.) 
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UNIVERSITY EXAMINATIONS 

INTRODUCTION 

WHAT IS A QUESTION BANK? 

WHAT CAN A QUESTION BANK OFFER? 

HOW MAY A QUESTION BANK WORK? 

CAN A QUESTION BANK BE USED FOR ALL 
SUBJECTS? 

CAN A QUESTION BANK MEET LOCAL NEEDS? 

STATISTICAL PROCEDURES 

COMPARABILITY AND MAINTENANCE OF 
STANDARDS 

TYPES OF QUESTION BANKS 


FUTURE FOR QUESTION BANKS 
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QV1SSTVM BANKS-U)^ IN INDIAN 
UNlVEBSiry f^AMINAXIONS 


1,0 INTRODUCTION 


Like fume-free e«re and the Kingdom of Heaven, queition 
banking is one of ^tixoee ideas which has great appeal but whi^ 
people do little about. In that respect, the position is little 
changed from 1971 when the Association of Indian Universities 
recommended strongly that the University Boards of Studies 
should ' frame* not only syllabus but also a * Bank of questions* 
for each of the courses. Many universities have since tried 
to expand their syllabi in various subjects into meaningful 
objectives and produced resource materials for students and 
teachers including evaluation materials. This is in keeping with 
the Report of the Rducation Commission (.1964-66 p. 220 ) 

V(rhere it is said * examination reform has become crucial to all 
progress and has to go hand in hand with the improvements in 
teaching'. The Radhakrishnan Commission said, * if we are to 
suggest a single reform in University Education, it is that of 
examination reform'. Dr. Amrik Singh strongly feels that 
priority should be given to improvements in teaching in 
universites and that it is wrong to suppose that changes in 
examinations will automatically produce desirable changes in 
university teaching. It is wise however to effect changes in bodi 
teaching and testing. Question banking is one of the strongest 
ideas that will attempt to integrate teaching and testing at both 
individual university's level and ^mongat various universities of 
the country. Some universities have gone ahead with Question 
Banking and many others^ are on their way to implement 
Question Banking as a pari of examination reform. At the same 
time a good number of universities is still either undecided or 
unaware of the potentialities of Question Banking. While 
individual universities/ institutions should be given encouragement 


1. Universities of Calicut, Poona, Rajasthan. 

2. Universities of Madras, Madurai, Mysore, Bombay. 
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aad assistance 1b tlie matter of Question Banking, a need to create 
an inltUl Question Bank in major first degree level siibjects 
useful for all universities in the country was strongly felt in 
terms of saving of time, effort and resource and also in terms of 
bringing about uniformity and standardisations of some kind* The 
Association of Indian Universities, in its Research Unit on 
Examinations, has taken it up as a major development activity, 
constituting initial Question Banks in First degree level subjects 
of Mathematics. Physics, Chemistry, Botany, Zoology^ History, 
Geogra|diy, Psychology, Economics and Commerce. In aU, 
there are about 60,000 questions/items in tiiese 10 subjects and 
these have been prevalidated, coded, classified and stored in 
the form of 8'^ x 5" cords in Xardex trays. Question banks, making 
full use of the accumulated experience of teachers and examiners, 
represent an important potential contribution to the process of 
examining. In our country, the Last decade or so has seen a 
marked desire for iinprov«iment in examining, perhaps character¬ 
ised chiefly by the evolution of improved school examinations by 
NCERT and other agencies in the various states and research 
into and concern for increased technical efficiency. The need 
at this time to consider and discuss such possible innovations 
as question banks is particularly strong in view of the increasing 
load which the existing pattern of examining must bear. The 
need is caused by rising numbers taking various examinations . 
widening range of examinable subject matter, the need for more 
frequent revision of syllabi, the breaking down of traditional 
subject boundaries, the movement towards a greater degree of 
involvement by teachers in the examining process and the 
continmng use of examinations for course evaluation, pupil 
diagnosis, educational' guidance and selection. 

At the same time the demands to examine with justice and 
to maintain and improve standards are as strong as they have 
ever been and may well become stronger as a larger proportibn 
of the population comes to be directly affected by examinations. 
Therefoze , the task of examining which already involves 
considerable educational and administrative effort will inevitably 
become more difficult and comolicated in the years ahead. All 
this adds vigour and point to the search for improved methods of 
examining and compels careful consideration of the role of 
" question banks". 
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The main objective of tiiie moaograph la to pMpoee a plan 
of action to build up Question Banks in our countr]f Birough 
co-ordination of different orgaidsations and universities avoiding 
duplication of efforts, piecemeal work and adhoc decisions of 
subjects. Some practical ways of building up and using Question 
Banks in the present context and thus maintain tiieir dynamic 
concept are suggested* 

2.0 WHAT iS A QUESTION BANK 

There is nothing revolutionary in the idea of a Question 
Bank. Stores or banks of questions have been in existence 
for a long time. Over tiie years, various universities., and 
examining boards have produced a large number of questions 
in many different subject areas; tiiese in effect constitute a 
series of banks from which questions are drawn for examining 
purposes and to which new questions are added from time to 
time. Similarly most teachers have developed their own 
questions for assessing the progress oftheir students. Teachers 
and examiners will be familiar, of course, with the task of 
drawing up suitable questions for purposes of assessment and 
teaching. Most of these questions were set largely on an 
intuitive basis because they looked "right" or bacause they 
seemed to work. In the hands of an experienced teacher or 
examiner this process has generated a consideYable store of 
questions that time has shown to be of merit. But such 
teachers and examiners have been few in number so that the 
less experienced have not had the same opportunity to develop 
this kind of judgement on the questions they devise and select. 

Question bank In the sense used in this monograph 
contains questions about which there is available some specific 
Information, usually of a statistical nature. In this sense they 
differ from question banks already built up by teachers and 
examiners. The main function of a Question Bank, if one were 
established would be to make available to teachers and 
examining bodies such questions together with associated 
information that-they might wish to use in preparing their own 
tests and examinations during or at the end of a course. 

Individual questions would he selected on the basis of experience 
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in ** f th9 ** tH® ficlil*** A n o th er 

® ifttMlioii wimld b® io kdp pxovld® Aft ®®tlnaAibi 
d®gr«« 9 t comparability ol «xaminatioi» pcodacod in 
IbJt w«y« 

Tbt pTOcosf of producing qucstiozw Cor a question bank 
might well result in a elarifiicatioi) of the aims and objectives 
of ^e examinations* This could lead to the specification of 
s;d 3 ject objectives in fairly precise and detailed terms. It 
would be a mistake to suppose, however, that a Question 
Bank would consist solely of those questions in which Ihe 
course objectives can be accurately defined and measured. This 
would be a narrow interpretatiiin of file way a Question Bank 
would work. A Question Bank could clearly be composed of 
all kinds of questions now being used for internal and external 
assessment, ranging from the open" or " essay type'* to file 
" closed" or "objective" type questione. It is necessary that 
a question bank must contain a large number of questione/ 
items in a particular subject. Going bsyond written questions/ 
ttAms there is no reason why other kinds oi tasks such as oral 
questions, dictation, musical passages, project topice and 
practical experiinents siu>uld not be stored provided that some 
quantitative evaluation of them will be made. Perhaps 
'Question Bank' is itself too restrictive a description of the 
enterprise, may be ' task' oi' stimulus' Bank is more appropriate 
a shift in emphasis which brings the conception very cloee 
indeed to two other kinds of educational bank bmitcd recently, 

" a resource bank" and " a syllabus bank." 

Ths Examination Bulletin 3^ describes the Question 
Banks as follows} 

"A BANK OF ITEMS ( QUESTIONS) OF KNOWN TECHNICAL 
VA LUES CAN BE BUILT UP FOR FUTURE USE. IT MIGHT, 
IN PRACTICE, PROVE OF VALUE TO ARRANGE FOR 
ITEM CONSTRUCTION TO BE A MORE OR LESS CONTlo 
NUOUS PROCESS. THE CONSTRUCTION OF WRITTEN 
PAPERS CAN THEN BECOME A MATTER OF JUDGING 
THE SUITAfilLITY OF ITEMS OF KNOWN TECHNICAL 
VALUE FROM A BANK OF ITEMS. ITEMS CAN BE 
WEEDED OUT AS OUT OF DATE OVER A PERIOD OF TIME. 

FURTH^ XT CAN BE SAID THAT NEW QUESTIONS SHOULD BE 
TRIED OUT M D STATISICAL EVIDENCES PCR ITS 
FACIUTY,DISCRIMINATION ASCERTAINED, IT IS ABSO* 
LUTELY NECESSARY THAT THE BANKS WOULD HAVE 

TO BE LARGE TO BE OF VALUE." _ 

I* ExamlnatlQn Bulletia 3"Uees An introduction to some techniquee ol 
tMmiBittg", HMSO , 19S4. 
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3. 0 WHAT CAN A QUESTION BANK OFF£R 


A quttBtion bank allows careful pretesting of questions 
from which advance information could be made available to 
examination boards; such information being of a statistical 
nature , sometimes accompanied by a description of the 
learning objective that the question was i intended to measure 
This description helps to decide retention, improvement or 
rejection of questions. The data would complement the 
judgment of examiners and teachers and not in any way 
supplant their necessary freedom to select their own questions. 
The practice of pretesting questions with representative samples 
would mean that teacher could compare the performances of 
his own students with those of the wider population, thus 
contributing to the overall maintenance of national standards. 

A carefully built up Question Bank can influence curriculum 
development. In building up a bank of questions some re¬ 
appraisal of what these questions were intended to reveal in 
terms of pupil learning might become necessary. A Question 
Bank could be composed then, in part at least, of questions 
produced to fit this specification of objectives. Each question 
would be related to a particular objective of learning and 
would therefore play a small but vital part in measuring ihe 
total achievements of the individual. 

The potential advantages are thus obvious. A Question 
Bank offers the chance (a) to produce and evaluate question 
on a relatively more effective basis; (b) to carry out 
concurrent comparability; (c) to match the examinations 
to the curriculum that is taught and not as tends to happen 
in the conventional setting the other way round. The Question 
Bank would act as a neutral educational and administrative 
service helping teachers and examiners to solve some of 
the problems with large-scale examining. 

4.0 HOW MAY A QUESTION BANK WORK ? 

The workability of Question Bank may depend on scope, 
scale of operations, the relationship with examining bodies 
and colleges and universites, financing and administration. 
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pMtiMlities of orguloiag fuoation boakv 
la pxmctlca. It ia ^opoacd to glvo fuidaaco rogarding tho 
variooa w«r« in whicli quaatloaa might bo dovalo|Md and uaad 
la a Questioa Bank, 

Univeraltiea that are selected to imidemeat escamination 
reform are to work in a coordinated way and help each other 
where ppaaible. 

There are two poialble procedures that suggest themselves* 
The first procedure would involve the use of questions and items 
already in exLstenee which examiners and teachers considered 
to satiefy their criterion of good " questions* These might be 
specific questions or they may be objective type questions* 

The second procedure would require that-questions were 
developed to measure as precisely as possible those objectives 
of learning that teachers and examiners considered Important. 
Such questiona/items would normally be devised at the end 
of a period during which some detailed reapprisal of learning 
objectives had been undertaken* the aim being to relate the 
questions to the specified objectives to build in validity. In 
an affiliating type of university* teachers drawn from different 
colleges teaching subjects at the same level are to be given 
a short orientation into areas of objectives* strengths and 
weaknesses of different types of items and questions and 
initiated into the scientific skill of item writing. An initial 
attempt may be spread over a period of 3 weeks in a workshop 
type course. 

Initially* one may tend to use the first procedure which 
is a good starting point. However* the second procedure 
ensures toat questions and items satisfy validity and it may 
well generate and train a number of teachers as item writers. 
Whatever procedure is adopted ( a combination of the two* is 
however advocated) * certain features are common, a) Question/ 
items must be tried out in the " field " or " pretested" in order 
to,obtain statistical data about how the questions/items behave 
with a sample, on the basis of the statistics* such questions/ 
items must also be retained* modified or rejected, b) Questions/ 
items must slso be tried under actual examination conditions. 
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la tills cats th« decisions nbonl ^Mstions wdoid be mod e after 
tile event end not belore ns In tlie cnso of pretestUig* But tiiis 
mny be vnlunble to yield operntioosl efanrncteriitics of questions 
stt|qdementin^ infbmntloa frimn pretiitlng- 

One mny suggest writing out on cnrds (designed for the 
purpose^ l^e questions to constitute the Baaik and stored* An 6 '*k 5'' 
sise card is Idenl for storing in Knrdem trays* Punched cards 
may be used and items/questions stored in computer. It may 
also be published ae a brochure"* each question with related 
statistical information, the learning objective tested eh Guide- 
Uaes may be given in &e matter of selecting questions from the 
brochure or from cards or from the computer. A question 
paper mi^y be ma.de with queetione from the bank to which 
some oti||ere may also be added. The items/questions that 
are drawn from the bank are to be analysed statistically and 
the data,gntered ^ appropriate columns in the cards. Items 
which are of objective type , when they constitute a test, may 
be marked and g-raded with the use of a digital computer. A 
computer programme has been developed that will mark the 
answer scripts of a test with objective type test items 
< Multiple choice) and make statistical analysis as weLU 

It would be difficult to imagine a Question Bank developing 
it full potential without making use of the technical and admini¬ 
strative advantages of a computer. The handling of large 
scale computations, the drawing of samples, the storage and 
retrieval of questions and related data argue the case for a 
computer* A question bank could however operate on a more 
limited scale, without being based on computer technology* 

For the use of teachers to make classroom tests ( continuous 
assessment) a brochure type bank of items that are to be 
pretested may be preferred. As the items/questions which 
are found to satisfy the criterion of facility, discrimination, 
difficulty level., could be made Into a bank of questions for the 
examiners to use in making "end of term" university examinations* 
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5.0 CAN A QUESTION BANK BE USED FOR ALL SUBJECTS 

Alt3i6ugH actual laglsties of operating a question bank 
for sample pretesting and statistical analysis might appear 
alike in most subjects , certain subjects probably lend them- 
seiKss TtioTit readily to the concept of question banking than 
o^ers. Mathematice, biology, physics, strength oi materials, 
electronics p might be thought more amenable to a detailed 
analysis of learning objectives than history, modern langiiages, 
psycd^ology, human geography. It is very difficult to spell 
objectivies if he latter subjects while it is very easy to do s 
in respect ot the lormer. Teachers may Uunk that question 
banking suits those subject areas in which it is thought easier 
to express questions in objective form. One might think that 
a question bank has very little to offer about subjects in which 
objectives are predominantly oral and skill-oriented though 
there is no logical reason why tape-recordings, scale exercises, 
prose passages, and the like should not be stored in a question 
bank. There are difficulties in applying an uniform concept of 
question banking to all subjects, though writing objectives is not 
a necessary precondition. Questions could still be used pretested, 
analysed statistically in situations where objectives bad not clearly 
been stated. Objective type examination has the advantage in the 
sense that accurate statistical analysis can be made, flssay 
type, problem type questions can also be used in the question 
bank even though statistical analysis in such cases tends to 
be crude. 

6.0 CAN A question BANK MEET LOCAL NEEDS 

In a few subjects like geography and history, it may be 
necessary to adopt questions to local needs. Questions in 
these subjects may be classified as a) those that are of 
universal significance, b) those that use local place-names to 
illustrate a situation but could be adopted for universal use and 
c) those that are not amenable to adoption in any way. It is wise 
to keen the percentage of questions in (b) and (c) to a minimum. 

The idea is that question bank can be used by many universities 
and that each univerity can work in a few predetermined areas 
or subjects so that duplication of efforts can be avoided. The 
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time, money and effort that goes into scientific production of 
questions for* bank in a country like ours necessitates such 
coordinated ways to do the job • It is also healthy to exchange 
the banks in different subjects from different universities. Such 
coordination and cooperation among the selected universities 
is strongly recommended so that these universities with the 
least expenditure of time, money and effort can produce and 
create item banks in different common subjects on a sharing 
basis . The Association of Indian Unlversties , with its 
Research Cell, is now quite willing to bring about this 
coordination and help produce a Central Bank of questions/ 
items in first degree subjects for use by all Indian Universities. 

7.0 STATISTICAL PRCX^EDURES 

It is proposed to give an outline of more basic calculations 
that may be used in the operation of a question bank. More > 
advanced techniques have been developjsd and it is probable that 
some modified form of these would be used in the operation of 
every large scale question bank. The main statistical technique 
is known as item analysis. In performing item analysis two 
main values are computed: (1) Facility Value ( F. V.) and (2) 
Discrimination Index ( D.l ). The first indicates the difficulty 
level of the question ( whether it is hard or easy ) and the second 
indicates the extent to which the question discriminates between 
good and poor candidates. A conventional item analysis is 
as follows: 

(a) Place all the completed scripts in order of merit of 
marks on the whole paper. Divide them into equal 
portions. Sixths or fifths are commonly used. 

(b) Take each candidate, paper and for each question in 
turn, count the number of candidates who gave each 

of the four/five possible answers in each of the sixths. 

( This is a multiple choice question with 4/5 possible 
answers). Thus all the responses are included in the 
analysis. This is useful because it may be possible to 
see whedier a particular answer is attracting too many 
responses and acting as what test constructors call 
a " distractor." 
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(c) A grid cgn then be coni|Aeted« Suppose B is the 
correct answer. 


Answer 

B 

B 

C 

B 


Total 

P6 


Top sixth 

1 

D 

1 

B 

1 

18 

0.67 

0.47 

2nd sixth 

2 


2 

D 

B 

17 

0.47 


3rd sixth 

1 


3 

1 

0 

18 

0 . so 


4tii sixth 

2 


3 

3 

fl 

16 

0. 50 


5th sixth 

3 

B 

4 

1 

5 

17 

0.24 

0.14 

Eotom , 
sixth 

5 

1 

3 

4 

5 

18 

0.06 


-1 : 

ToUl 14 

- * 

43 

16 

15 

10 

106 

0.41 



P6 proportion of total number of candidates in each 
sixth who obtained the correct answer. 


P3 proportion of total number of candidates in each 
theird who obtained the correct answer. 

(d) The grid shows 43 out of 106 answered the question 
correctly. Thus the question has a facility value of 
43 V 100 = 41%. This is on the low side for an 

106 

attainment examination. 

(e) The D.l. value is computed in many ways. It is 
obtained by (i) sabtractlBg tile F* V. of lower third from 
the F.V. of upper third. 

(ii) subtracting the F.V. of lower ability group from 
the F.V. of the higher group (Top 27% of population^ 
Lower ability groups Bottom 27%. 
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<f) Baaed on thirds, D.l. of this items 0.57^0* 14=0.^3 
Ideally we would aim to have li D.l. value around 0. 8. 

In practice^ower D.l. values of around 0.6 are 
acceptable • 

The value of these statistics lies in theit capacity to 
identify unsatisfactory questions. A high F. V. means low D. l. 

Lew D.Lmay bs also due to low F.V. This means that the item is too 
easy ( former case) or too hard ( later case) to bo afale to 
discriminate good from bad ones. A question may be ceafeeiag 
or ambiguous. As a rule, the distractors must bo accounting 
for a greater number of lewer ability group than higher ability 
group. An option diat la not the correct answer should 
negatively discriminate. 

When one is attempting to sample a defined body 
of content as in a subject examination, it is usually 
undesirable to drop questions just because their fV. and 
D« L values are low. Most examiners will tolerate Ipw 
value on the grounds that these will csutrlbiits to better 
sampling and enable high content validity to be secured. 

Generally speakingahigh D.Lvalues are required when 
it is necessary to distinguish different levels of achievement 
(Selection) . Where however the examination is designed 
to provide feedback information for the purpose of course 
evaluation, a high D. I. value is of less irportance. 

One important point that is to be borne in mind is the 
difference in statistics that would arise for pretesting and 
actual examination conditions since it will be very difficult 
to get a representative sample. 

8.0 COMPARABILITY & MAINTENANCE OF STANDARDS 

Careful pretesting of 'questions on represeotativs samples 
of the relevant population would bedntrihsic to the operation 
of a question bank. Each question tested in this way would 
have a wide national significance and would be Used by 
teachers and examiners as a basis for comparision between 
their own candidates and those of the pretest sample. Teachers 
would compare the item statistics calculated for their own 
group of students against those for the pretest sample* la 
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this way the qiv^scion bank can provide evidence to teach era about 
the performances of their students relative to others following 
similar courses and would thereby help to maintain standards 
at a reasonably constant level. The proportions of the pretest 
sample in each grade band who correctly answered the various 
questions would be known» This can help to predict their 
grades in the examination reasonably correctly, 

9, 0 TYPES OF QUESTION BANKS 


The types can range from a collection of past examination 
papers of a university or class test items of a teacher to the 
most sophisticated stored items/questions in the computer, 

(1) A collection of items/questions published in the form of a 
booklet/brochure may be made available together with those 
prepared by a group of teachers after analysing objectives 
of learning various content areas. Initially this may be used 
for pretesting and collecting statistical evidences, (2) Once 
a reasonable statistics is obtained, each item/question is 
looked at critically and retained, improved or rejected. Those 
that are improved and retained will be transferred on to cards 
especially designed for the purpose, for use by examiners to set 
end of term or university examinations. (3) Those Items/ 
questions that are retained can also be stored in the memory 
of a digital computer. Whatever type is chosen^ the dynamic 
concept of question banking must be kept up. 

10.0 FUTURE OF QUESTION BANKS 

This explains the concept of a question bank, to describe 
how it might work and to explore some of the issues involved. 

It is hoped that this would be discussed by every University/ 
Examination Board/Institution trying to build up question 
banks in our country and evolve methods that would be feasible, 
taking into account local conditions and constraints. Evidence 
collected on the operation of Question banks in the next few 
years will certainly add another dimension to-modern examining 
procedures in our country. 


SECTION II 


QUESTION BANKING - OUTCOMES 
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QUESTION BANKING - OUTCOMES 


It is indeed necessary for us to look at some of ther 
possible outcomes of Question Banks. Outconies are 
very many and they depend on how teachers, students, 
administrators and universities use the question banks. 
As outlined earlier, the Question Bank consists of a 
large number of different types of questions/items in 
every subject, with known technical values. 

Some technical values of a question/item are : 

i) The content area/topic with which it is concerned. 

ii) The intellectual ability that the question/item 
demands of the student. It may be knowledge, 
comprehension, application, analysis, synthesis 
or evaluation. 

iii) The time required for answering. 

iv) The type of question/item whether selection 
or supply. 

v) Marks allotted. 

vi) Facility value or difficulty index 

vii)Discrimination index. 

1. The most important outcome is the availability of a 
large number of questions/items in both selection 
and supply types. Selection type includes : 

a) constant alternative T/F 

b) multiple choice 

c) multiple facet 
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d) matching <simple and compound) 

e) rearrangement 

and supply type includes : 

a) simple question 

b) completion 

c) short answer question 

d) long answer question 

e) problem solving 

When the questions/items are carefully selected 
from sources of 

1) specially conducted Question Banking Workshops 

2) item writers/teachers and 

3) past examinations 

it is very likely &at they can be made o measure 
important abilities In a content area ai for various 
content areas. In Question Banking Workshops, it 
will be possible to clarify outcomes of learning a 
content area and expose to valid ways of testing the 
achievement of these objectives of learning systema¬ 
tically so much so that questions/items generated 
from these workshops will be of very good quality. 

A good number of teachers have already been trained 
in the science of item writing and they are also 
involved. Such questions/items are prevalidated 
with a checklist of criteria for good quality questions/ 
items and some technical values namely (i) to (v) 
above are incorporated* 
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Teachers in colleges/universities teaching 
these subjects are now helped with a collection of 
questions/items prevalidated and they can initially 
use them for class tests. Various universities can 
use these questions/items for the end of semester 
or year examinations. 

Teachers can analyse the responses of students 
and begin to undertake the professional tasks of test 
and item analysis and work out facility value and 
discrimination index of these and make inferences 
about teaching, student's learning etc. 

II. Teachers and examiners will spend very little time 
in making up class tests and university examinations 
and spend the,time thus saved in analysing responses 
and improvement of questions/items. Every depart¬ 
ment in a college can arrange to store items/questions 
in specially designed 6"x5'' cards and store them in 
Kardex trays. The Controller's office can adopt a 
similar procedure or even store these .questions ’/items 
in the memory of a digital computer. There can be 

a considerable saving of time and effort in making 
up question papers. If the initial questions/items 
bank is fairly large, (atleaat 20 to 30 times the number 
of questions in a single paper) then the choice of 
questions/items is meaningful. 

III. Teachers and examiners will be continuously Involved 
in creating more and more items for Inclusion, thus 
keeping themselves professionally involved. 

IV. There can be a backwash effect on teaching. The 
nature of questions/items being so varied, will 
impose innovative methods of efficient teaching and 
effective learning. 

V. Question banks offer to carry out concurrent compa¬ 
rability. The same items/questions at the same 
level returning the same statistics of FV and DI in 
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different universities will ensure comp>arability 
of standards. A third university that returns very 
poor IV and for the same items» can be rightly 
said to have low standards. It -is ^us possible to 
provide a definition of standards and question banks 
if carefully designed and operated, will help maintain 
' standards'. 

VI. Question Banks if carefully built up can influence 
curricular development. A meaningful way of 
interpreting syllabus, writing objectives, building 
valid questions/items are some of the possible 
gains. 

VII. This may well generate a number of teachers as item 
writers. They will develop scientific skills not only 
in item writing,_but also in prevalidating them. 

VIII. If FV and DI are known for all items/questions, 
teachers can use Question Bank for making diagno¬ 
stic tests, mastery tests and criterion referenced 
tests etc. 

IX. Careful pretesting of questions on representative 
samples of the relevant population would be intrinsic 
to the operation of question bank.-)Each question 
tested in this way would have a wide national signi¬ 
ficance and would be ^ised by teachers and examiners 
as a basis for comparison between their own candi¬ 
dates and those of the pretested sample. Teachers 
would compare the item statistics calculated for 
their own groups of students against those from the 
pretested sample. In this way, the Question Bank 

can provide evidence to teachers about the performance 
of their students relative to others following similar 
courses thereby helping to maintain standards at a 
reasonably constant level. The pxoportions of the 
pretest sample in each grade band who correctly 
answered the various questions would be known. 

This can-help to predict their grades in the examina¬ 
tion reasonably correctly. 
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Teachers, examiners and Boards of Studies 
members will be quite prepared to restructure 
their question papers in various subjects including 
more and more of objective and structured short 
answer questions, thereby ensuring increased 
content validity and overall reliability of their 
examinations. It is very lilcely that written papers 
in various subjects may be made up of a Part A 
objective (ZO to SO items for ^ hour} Part B short 
answer structured questions (10 to 15 questions 
for 2 hours) and Part C long answer/problem solving 
questions ( 1 to 2 questions for j hour). 



SECTION ni 


levels and forms of question banks 
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LEVELS AKD FORMS OF QUESTION BANKS 


The basic idea behind question banking could hardly 
be simpler* A library of questions/items of all descriptions 
is put at the disposal of teachers and examiners wishing to 
set either terminal examinations or periodical classroom tests 
or in the case of examining boards written, external semester 
or yearly examinations* Undoubtedly the uses to which banks 
of questions/items pan be put will determine the level and form 
of them. There can be many criteria for classification into 
levels while the forms are usually dictated by the manner 
in which questions/items are collected and stored* It is necessary 
for teachers, examiners and administrators to know clearly 
the different levels and forms of question banks, (in different 
subjects, for universities')will take, in order to bf able to 
realistically use them. In fact, there is more to question 
banks than first meets the eye; it can be at the simplest 
level^a collection of questions/items from different sources 
while at its most complex, a set of prevalidated, coded 
items/questions of known technical statistical values (FV k Dl) 
stored in the memory of a digital computer capable of being 
retrieved 

One way of classification is to base it on the level 
of sophistication* This may be Level Zero, A simple 
collection of questions/items of all kinds ranging from selection 
to supply from different sources and put into a file. At this 
level, the question bank has very little to offer except that there 
is a collection of questions/items* Not even an atempt to relate 
the questions/items with the syllabus, is now made* 

Level One 


At this level, it is not a collection of questions/ 
items only. They are reviewed by a group of teachers who 
pass judgment about the questions/items with the help of a 
checklist of criteria for prevalidation along with an expert 
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on evaluation. Questions from different sources like 

1) specially conducted question banking workshops; 

2 ) teachers/item writers who had been trained in eva¬ 
luation methods; 

3) past examination (apers 

are prevalidated and technical values about the questions/ 
items are given, 

Level Two 


At this level, there is the further provision that they 
are stored in cards (8” x 5") specially designed to contain one 
question or one item per card along with technical values. 

In addition there are columns for entering FV & DI of file 
questions/items after use. After a period of time there will be 
available questions/items with technical values and characteristics 
like FV & DI for use by paper setters and teachers. A card 
designed for initial question banks set up by AX U is given 
at the end of this section. 

Level Three 


At this level, it must include all that is included at 
level Two together with the provision that these questions/ 
items are coded and stored in the memory of a digital 
computer. It must be possible to retrieve any question/ 
item and a neii^s of them to constitute a class test or a 
university' examination. One main advantage is that if we are 
thnking of obtaining different forms of the same objective type 
test items to ensure chat all students taking the examination, 
ant srer the same items arranged in different ways so that 
they cannot communicate with each other, it is possible 
through the use of the computer. 

The question arises as to what must be the proper level 
at which question bank must be planned and set up in an individual 
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university* Level Three must be the level at which an 
individual university^ having an examination unit and 
potential of using computers in examinations must start « 
Question Bank* Alternatively, the university cai attempt 
at Level Two with the provision to update it to Level Three 
at a future date* 

There could be yet another basis for classification* 

This may be Level A* An individual member of staff teaching 
a particular course, collects a good number of questions/items 
of all types (selection consisting of constant alternatives, 
multiple choice, multiple facet, rearrangement and matching 
and supply type consisting of completion, simple question, 
short answer, long answer and problem solving) for use in 
class tests* He may try to make on his own^a few questions/ 
items to match instructional objectives already written by 
him* He can also undertake systematic item analysis and find 
out FV 8c DI of all the items/questions* He may use it to improve 
his own teaching and also to diagnose weaknesses of learning* 

Level B A department in a college with ( alt its staff constitutes 
initial question banks in their subjects collecting questions/ 
items from 1) past examination papers^Z) the staff of the depart¬ 
ment and 3) specially conducted workshops to produce requisite 
number of items/questions. The ^oup would prevalidate these 
questions/items with a checklist of criteria and perhaps store 
the questions/items on cards (8" x 5") in Kardex trays. They 
may undertake to do item analysis. 

Level C At this level, a university builds up a question 
bank with the help of teachers from various affiliated colleges 
coming together* This may be done 

a) by collecting questions/items from different 

sources that of past examinations, teachers, item 
writers and participants in specially conducted 
question banking workshops; 
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b) by prevalidating questions/items with the help 
of a checklist and by groups of teachers; 

c) by storing them on 8" x 5^' cards in Kardex trays, 

A systematic plan to get item analysis details incorporated 
is made out and university examinations are set by paper 
setters nnaking use of the question bank with Table of 
Specifications, already prepared. 

Level D This is the level at which the Association of 
Indian Universities has undertaken to constitute initial 
Question Banks in different first degree level subjects* To 
start with Questions/items were collected from a) specially 
conducted Question Banking Workshops at various colleges/ 
universities; b) teachers and item writers irom all over 
India who were trained in Evaluation methods and c) past 
examination papers of various universities. These questi^s/ 
items were prevalidated by groups of teache^rs and ev ainat ioT 
experts using a checklist of criteria for pre validation and 
all technical values are given. Ihen these questions/ltoms 
are put on cards (8^' x 5'0 and stored in Kardex trays, llioss 
questions/'items are also printed in the form <»f books and 
made to universities. Cards in Kardex trays are 

also available as a system. A syatematic 'field trial* will 
be done and these questions/items that are used in university 
examinations item analysed. Details of item analysis will be 
incorporated in these cards. 

Forms of Question: Banks 


Depending upon the levels at which these are built 
up and also upon the uses to which they are likely to be 
put, there can be different forms. In its simples form, it can 
be a file, each question/item, type written and <:la8sified 
into dfferent types in serJea content topics. It can be in the 
form of 8" x 5'* cards simply carrying one question/item 
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per card and stored* An improved form will be the same 
8" X 5” cards vdth lot of printed columns to enter the 
various technical values like code number, subject, topic, 
date of making, type of question, objective tested, level of 
question, time for answering and marks allotted. At the 
back of the card there should be provision to enter FV/Dl 
a.s fthown in the specimen card elsewhere given in this 
Monograph. These can be stored in Kardex trays. In this 
form this can exist in the Controller's office for use in aetting 
up examination papers. In another form, these Questions/ 
items can be coded and stored in the disc pack of the digital 
computer (about 1,20, 000 questions/items can be stored on a 
single diBcpack,). Another usual form is to print it as a 
book adopting two column single space (this can have more 
number of questions/items per page) giving either all the 
questions/items or a 20% sample of the total number of 
questions/items. Usually for purposes of fanrdliarity and 
practice teachers and students will be given questions/items 
in the form of a book. Such books one in eaqh of these ten 
subjects, are being brought out by the AIU, making available 
to teachers and examiners somewhere around 5000 to 6000 
questions/items in every subject. 



ASSOCIATION OF INDIAN UNIVERSITIES 



Front fllda of tlM QuMtloD/IteiD card 











Back aide of tba QuoaUon/ltviB card 
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PLAN OF ACTION TO BUILD QUESTION BANKS 


Banka or pools of different kinds of questions/ items 
in various subjects can be built up for future use. The 
question/item bank may be stored with (a) an individual 
teacher teaching a particular subject in a college at a 
particular level, (b) a group of teachers in the same college/ 
institution teaching the same subject to different sections of 
the students at the same level, (c) a group of colleges under 
a university and finally (d) a group of universities or all 
the universities^ 

The Research Cell of the Association of Indian 
Universities as a part of its Development Project, under¬ 
took to build up initial question banks in 10 first degree 
level subjects of Mathematics, Physics, Chemistry, 

Botany, Zoology, History, Geography, Psychology, Commerce 
and Fconomics. A comprehensive plan of action was evolved 
and the project executed in stages with time scheduling. 
Given below is the entire planning of this development 
project that has produced nearly 60, 000 questions/items 
all prevalidated and stored in specially des'-^Tied 8”x5" 
cards in Kardex trays: 

QUESTION BANKING PROJECT 


PLANNING 


S, No. STAGES 

1. STAGE I : DECIDING PATTERN OF QUESTION 

PAPERS IN THE SUBJECT"? 


2. STAGE II : COLLECTING QUESTIONS/ITEMS 

FROM PAST EXAMINATIONS k OTHER 
TEACHER/ITEM WRITER SOURCES 

3. STAGE III : PREPARING DRAFT QUESTIONS/ITEMS 

THROUGH WORKSHOPS oN QUESTION 
BANKING IN INDIVIDUAL COLLEGES/ 
UNIVERSITIES 
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4. STAGE IV : SCREENING, SE1J:CTING QUESTIONS/ 

ITEMS AND CONSTITUTING INITIAL 
SA2«XS 

5. STAGE V ; PREPARING QUESTIONS/IT8:mS ON 

specially designed carijs and 

STORING IN KARDEX TRA YS 

6. STAGE Vl : SAMPLING 20% OF THESE QUESTIONS/ 

items reconstituting into 
cyclostyled forms for field 
trial 

7. STAGE Vn : FIELD TRIAL OF QUESTIONs/lTEMS 

THROUGH TEACHERS 

8. STAGE Vin : INCORPORATING ITEM/QURsTION 

behaviour CHARACTERISTICS 

ON CARDS 


DESCRIPTION OF TASKS IN STAGES 


STAGE I 


DECIDING PATTERNS OF QUEST ION 
PAPERS IN THE SUBJECTS (MiC THFMATTrs 
PHYSICS. CHEMISTRY. BOTANY 200 ir)GY ' 

psychoiogy. history! geog raphy! ^ 

COMMERCE AND ECONOMICS) A T FIRST 
DEGREE LEVEL 


FVeparing^ a suggested pattern of question papers with 
the help of initial work done at Calicut (years ago) and 
Madras University (recently) by their Boards of Studies 
and addressing a 2o% sample of mainstream Universities, 

Periodically consolidating the replies from these 
universities. 


* summitry end conclusive patterns of 
question papers in these subjects. 



43 


4, Deciding on the number and types of questions/ 
items to go into tbe Initial Question Banks in 
these subjects. 

5, Preparing a report of Stage I. 


STAGE II - COLLECTING QUESTIONS/ITEMS 

FROM PAST EXAMINATIONS AND 
OTHER TEACHER/ITEM WRITER 
SOURCES 

II. A. PAST EXAMINATIONS 


1. Preparing a circular letter to all the mainstream 
universities to send the question papers in these 
subjects in the last 10 years together with syllabus 
copies. 

2. Consolidating the replies and collecting these 
question papers. 

3. Classifying these questions on a preprepared 
categorisation ^in terms of both content and 
outcomes). 

4. Selecting i>ool8 of questions/items from these in 
individual subjects. 

5. Typing out on cards (specially designed 8"x5"). 

6. Storing these items/questions on cards in 
Kardex trays. 

II. B. ITEM WRITER SOURCES 

Preparing a list of teachers/item writers in these 
subjects and writing circular letters to all of them 
to send as many items/questions as they can within 
a stipulated time. 


1 . 
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2. ConsoUdatiiig the replies and collecting these 
que 8 tion s / items • 

3. Classifying these qnestions on a preprepared 
categorisation (in terms of both content and 
outcomes). 

4. Selecting pools of questions/items from these, 
in individual subjects. 

5. Typing out on carde (specially designed 8"x5'’). 

6. Storing these items/questions on Kardex steel 
trays* 

STAGE HI - PREPARING DRAFT QUESTIONS/ 

ITEMS THROUGH WORKSHOPS ON 
QUESTION BANKING IN INDIVIDUAL 
COLLEGES/UNIVERSITIES 

1, Preparing a list of colleges/universities in the 
country eager to conduct Question Banking Project 
Workshops to produce questions/items in these 
subjects at the first degree level. 

2, Preparing course details/handouts/materials for 
use during these workshops. 

3, Conducting Workshops at various places as in (1) 
above. 

4, Collecting questions/iteris prepared by participant 
teachers for use in Question Banks at the Association 
of Uidian Universities. 

5, Classifying these questions/items on a preprepared 
categorisation (in terms of both content and outcomes). 


6 . 
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Selecting pools of questions/items from these in 
individual subjects. 

7. Typing out on cards (specially designed 8*'x5"), 

8. StoFBig these questions/items on cards in Kardex 
Steel Trays. 

STAGE IV - SCREENING, SELECTING QUESTIONS / 

ITEMS AND CONSTITUTING INITIAL 
BANKS 

1. Preparing a draft of questions/items from different 
sources 

a) past examination papers II A. 

b) from teacher/item writers II B. 

c) from participants in Workshops on Question 
Banking III. 

2. Screening, selecting with the help of a checklist. 

(1) In respect of (a) and (b) above, small groups of 
4 to 5 specialists will be invited and asked to 
go through questions/item 8 and select with the 
help of a checklist of criteria for good questions/ 
items. 

(2) In respect of (c) above, the participants themselves 
will be engaged in group discussions to select 
from among those produced during the Workshops, 
questions/items that satisfy the checklisted 
criteria, 

STAGE V - PREPARING QUESTIONS/ITEMS ON 

SPECIALLY DESIGNED CARDS AND 
STORING IN KARDEX TRAYS 

1. Deciding on the design of cards (8"x5''); 
quotations; finalising order; inviting quotations 
for Kardex Steel Trays. 

2, Obtaining the requisite number of cards for the 
proiect/number of Kardex tray* I® store. 
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Typing out questione/items on cards. 

4. Storing them in the Kardex trays. 

STAGE VI - SAMPLING 20% OF THESE QUESTIONS/ 

ITEMS RECONSTITUTING INTO CYCLQ . 
STYLED FORMS FOR FIELD TRIAL 

1. Selecting the number of questions/items (random 
20% sample). 

2. Checking on the coverage of content/outcomes; if 
possible putting them into Sections (Unit test 
materials). 

3. Cyclostyling these. 

4. Assembling in the form of booklets. 


STAGE VII . FIELD TRIAL OF QUESTIONS/ 

ITEMS THROUGH TEACHERS 

1. Despatching these booklets with unit test materials 
to different universities to try out. 

2. Performing individual item analysis for FV and DI 
as and when the formats of performance are 
received. 

3. Consolidating these item analyses 

Preparing individual and summary data of item 
analysis of these items. 


4. 
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STAGE Vni - INCORPORATWG ITEM/0UESTION 

BEHAVtOUR CHARACTERISTICS 

1. Comparing individual on cards/suiranary charactar- 
iatics and analysing. 

2* Typing out summary characteristics of FV/DI on 

these items. 


A plan of action to build Question Banks for a 
university for use in preparing end umrersity examina- 
tions was drawn up for use by certain universities wanting 
such a plan. A few universities (Calicut* Madras, Madurai, 
Mysore, Gujarat, Calcutta) have already started putting 
Che plan into action. Given below is the Plan of action 
suggested to and accepted by Madras University, to build 
up Question Bank in PUC/Undergraduate subjects: 

1. The individual Boards ofStudies in different subjects 
in which initial Banks of items/questions are required 
must decide on the pattern of their examination paper. The 
pattern will include details like types of questions, time 
and mark allotment, as decided by the Syndicate in its 
decision on the reform of the structure of question papers. 

A typical pattern can be somewhat like this. 

Each paper is made up of 

Part A Objective type items - 1 5 to 30 numbers, 

15 to 30 minutes and 15 to 30 marks. 

Part B Short answer questions - 10 to 15 numbers, 
40 to 120 minutes and 40 to 60 marks. 

Part C Ix>ng answer questions - I to 3 numbers, 

30 to 90 minutes and 10 to 30 marks. 


The pattern can differ from subject to subject. An important 
feature .of this pattern i's that compared to present practice, 
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it increases the,number of questions to be answered quite 
significantly. With fop,ty odd questions it is possible to 
cover the syllabus adequately and to include all of the 
important objectives which is not possible when students 
answer only 5 or 6 questions. The general recommendation 
is that short answer questions and multiple choice questions 
be used substantially and increasingly and that essay ques¬ 
tions be used sparingly. 

Long answer questions are useful when the aim is to 
test the ability of students to express their ideas in appropriate 
language to develop an argument or a sequence of ideas. 

Their use is Indicated particularly in language examinations 
where, of course, they should not be overused . For most 
purposes, other types are more efficient. And with long 
answer questions, it is difficult to achieve reliable marking. 

It is necessary that every effort must be made to 
train staff and potential paper setters in item writing. 

Good quality items/questions must be identified and 
selected by groups of teachers. It is also possible to 
start question banking projects in individual colleges in 
different subject areas involving 4 or 5 teachers per subject 
and train them to produce good quality questions/items in 
a 10 day Workshop on Question Banking. 

2, The Boards of Studies after having decided on the 
pattern of the paper must further decide about the initial 
question bank in their subjects. Details like the type and 
total number of questions/items, modus ope rand i of going 
about making these items/questions, storing and using the 
same for setting examinAtions papers are to be finalised. 

It is suggested that the initial question bank must be atleast 
20 times as big as one examination paper (i.e. each subject 
question bank must contain initially 20 long answer questions, 
400 short answer questions and 400 multiple choice questions, 
if the paper consists of 1 long answer question, 20 short 
answer questions and 20 multiple choice items). 
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The modus operand! can be somewhat like this : 

Available questions/items (those with individual 
teachers and. from past examination papers of the 
university) can be collected. A group of subject 
matter specialists together with an evaluation 
expert can go through these questions/items with 
a checklist of criteria for good quality questions/ 
items to prevalidate them. Those questions/items 
that are found satisfactory can be classified and 
stored on cards for use. 

A better and rational procedure (this can be in addition 
to (a) above) is to get a group of subject teachers to 
assemble for a Question Banking Workshop to orientate 
them into srcss like objectives, strengths and weaknesses 
of different types of questions/items and initiate/draw ' 
into item writing skills. A draft copy of these questions/ 
items must be given to every teacher participating in 
this workshop to go back to his/her college and put 
these questions/items for systematic field trial. Item 
statistics like Facility Value (FV) and Discrimination 
Index (Dl) must be worked out. A practical possibility 
will be that a 20% sample (4 long answer questions. 

80 short answer questions and 80 multiple choice items) 
can be cyclostyled and given to teachers/students for 
practice and familiarity. These items/questions can 
therefore be analysed for their FV and DI. Over a 
period of 4 to 5 years, all the items/questions would 
have been field tested. Every year 10 to 20% items/ques¬ 
tions must be added. 

In regard to storing the rest of 80% (or all of them) 
the Controller's office can do this by getting items/ 
questions transferred on to specially designed 3"x5" 
cards and stored in Kardex trays. Later, even 
storing in a computer memory disc pack is possible. 
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d) In regard to using question bank for paper setting 
the paper setter can be asked to come to the office 
(bniverslty controller) with a Table of Specifications 
prepared by him/her: items/questions can be selected 
from the bank and handed over to the controller in 
a sealed cover. 


This way the whole process of making an examination paper 
is made less time consuming and more effective. To 
summarise: 


STAGE 


TARGET REMARKS 


1) Board of Studies Meeting 
to decide pattern of ques> 
tion papers, decide Modus 
operand! for question 
banking 

2) Initial Banking with past 
examination papers and 
other teacher sources 


1 day The Board Members 

may be initiated into 
the issues involved 
in the presence of an 
evaluation expert 

2 days Group discussion with 

a checklist of criteria 
for prevalidation 


3) Question Banking Work¬ 
shop for teachers (S to 
6 teachers per subject) 


10 days 


Through lead lectures, 
illustrations^ practice 


sessions and group 
discussions, to produce! 
the requisite number of 
questions/items for 
the bank 


4) Screening, selecting 
questions and consti¬ 
tuting a bank 


2 days Group discussion (A 

panel of subject expertsl 
and a few evaluation 
experts) 


5) Preparing questions/ 1 month Controller's office 

items on cards and 
storing in Kardex trays 


6 ) 20% sample of questions/ l month - do - 

items cyclostyling and 
distributing to individual 
colleges 
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7) Field Trial of 
questions/items 


8) Incorporating item 
behaviour charac¬ 
teristics with the 
items on cards 


1 acade- Individual teachers 

mic 

year 


Plan of action for constituting Question Bank 

in a College 

If Question Banking is to be introduced as a part of 
Fxamination Reform at the college/institution levels this 
could assume a slightly different role in terms of the college/ 
institution. One may look at the whole concept of Question/ 
Item Banks in different subjects in various departments as 
something that will be useful to teachers 

a) to make up class tests, assignments, tutorials 

b) to post validate questions/items in terms of 
responses by students to these over a period 
of time 

c) to build up their professional skill in writing 
different kinds of questions/items 

d) to modify their teaching methods and adopt more 
and more effective methods of teaching/learning 

e) to diagnose difficulties in student learning 

On the basis of first hand experience working with a 
good number of colleges (autonomous and others on their 
way to ) and institutions, the following plan of action is 
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suggested for implementeticns 

Stage I Meeting of ell heads of Departznents to discuee 
the number of subjects to be handled every year, 
the modus operand! to constitute an initial ques¬ 
tion/item baAk, to decide the pattern of question 
paper, to decide the number of questions/items 
in these, to decide the form of question bank, 
the form of storing, using and making it alive 
and dynamic 

Stage I* 1 day, preferably before vacation. 

Stage II Each Head of Department will discuss with their 

staff the total involvement of all staff in the process 
of constituting Question/Item Bank in the subjects 
decided to be taken up for Banking. Three different 
sources can be thought of : 


a) from past examinations papers and class tests 

b) from teachers trained in evaluation previously 

c) Question Banking Workshops conducted to 
train all staff of department 

Stage II, a) a vreek 

b) a mondi or the entire vacation period 

c) 1C days at the start of an academic year 

Stage tll Prevalidation of que stions/items with a checklist 
of criteria for good quality questions. Separate 
small groups can go over these que stions/items 
and with the help of the checklist, prevalidate 
and incorporate changes if any in addition to 
giving 


a) a bibliographical coding 

b) the topic area being tested 
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c) objective being testeH 

d) time for answering 

e) marks to be allotted 

f) correct answer or main points of answer 
Stage III. 10 days. 

Stage IV Typing out on cards (8"x5") and storing in Kardex 
trays 8"x5" specially designed cards are to be 
made available as also a Godrej Kardex tray. 1000 
cards would cost Rs. 100/- and tray for storing 
5000 cards would cost Rs. 500/«. 

Stage IV, One month. 

Stage V Putting the questions/items on Field trial. This 

may be done in the form of class tests/assignments/ 
tutorials. In the case of class tests, the items/ 
questions are analysed for F, V, and D, I, and these 
are incorporated in the cards carrying these items. 
In the case of tutorials/assignments, taachers can 
use the analysis of results of responses to know 
about teaching efficiency and learning progress; 
to make good in teaching as well as to diagnose 
weaknesses in learning and remedy. 

Stage V. one year. 

Stage VI Meeting with sUff and Head of Department to 
decide issues of keeping the bank alive ^nd 
dynamic. 

Stage VI , One day preferably at tiae end of 
academic year. 
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Atleast a dozen colleges have already instituted a 
Question Bank in every del>artment. There are perhaps 
many more who are on their way to doing so. 

AN ALTERNATIVE 


PROPOSED OUTLINE FOR TEACHER COLLEGE PARTICI- 
TION IN THE PREPARATION A ND EVALUATION OF MULTI¬ 
PLE-CHOICE QUFSTIONS/ESSAY QUESTIONS (SHORT 
LONG ANSWER) FOR QUESTION BANKING 


STAGES 


PEOPLE INVOLVED ASS OCIATED 

ACTIVITIES 


1) Syllabus 1. 1 Committee of educa-(a) Training for 

tional and employer Question Writers 
representatives 


Z) Examination 2. 1 Panel drawn from 
Specification the above + 

(or blueprint) 'specialists' 


2, 2 Panels of college 
teachers. Because 
examination speci¬ 
fications are 
detailed syllabus 
interpretations, 
colleges would be 
invited to submit 
their own versions 
for all or part of a 
specification 


3) Preparation of 3 .1 College teachers + 
Draft Questions other 'specialists' 
(following 
blueprint) 


To be successful, 
the system indicated 
requires continuous 
participation of 
teachers. It requires 
their knowledge, 
skill and experience 
for question wri ting 
and their cooperation 

in question! evaluation 

Successful question 
writers need training 
and guided experience 
It is anticipated that 
the Colleges of 
Education will contri¬ 
bute towards provi¬ 
ding this training 
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4) SttbmisBion of 4,1 College teachers 
Draft Ques* + other 

^ tions to exami - 'specialists' 

nine Board 


5) Preparation 5,1 Panel as for 2.1 
of 'Question- 

sets for Pre~ 
testinfi 

6) 'Pre- testing 6,1 College teachers 
of 'Question- 

sets' 

7) >5ttbmi" 

88ion of 
PrC" test 
data + 
comments 

8) Question 
Evaluation 
using field 
data 

comments 


7. 1 College teachers 


8,1 Examination Body 
(+ Panel as for 
2 . 1 ) 


(b) Syllabus Revisicm 


The preparation of 
examination speci¬ 
fications and the 
writing of questions 
to match specifica¬ 
tions provides an 
excellent means for 
a detailed systematic 
inspectioin helping 
to identify areas 
where syllabus 
wording and syllabus 
content require 
amendment. Thus 
teachers engaged in 
question Writing will 
also be in an exce¬ 
llent position to 
forward their views 
on the syllabus 
itself. 


9) Question 9,1 Examining Body 
'Banking' i, e. 

Classification. 

Recording 
Evaluation 
data and 
Question 
Storage 


10 )Selection of lo. 1 Examination Panel 
Evaluated (Chief Examiner + 

Questions for advisers) 

Examination 
Papers 



SECTION V 


STORING QUESTIONS/ITEMS 
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STORING QUESTIONS/ITEMS 


Storing questions/items is relatively simple but 
tremendously important. In a university set up when the 
Controller of Examinations is to get papersetters in different 
subjects to select questions/items from a bank of cards in 
Kardex trays, a certain amount of secrecy is involved. 

Initially the questions/itoms can be neatly typewritten on 
a side of the 8" x 5" card which carries in addition to the 
question/item, certain details called 'technical values' of 
the question/item. A sample question/item bank card is shown 
below. As the number of cards goes on increasing as 
well as the number of subjects in which question/item banks 
are built, it becomes increasingly difficult to think in terms 
of storing in Kardex trays; An enormous extent of space, 
amount of money and effort are required to maintain and 
use the banks. 

As one might expect, it is when the number of 
variables comprising the specification or sheer numbers 
increase ^thenLa manual system loses efficiency. A computerized 
systejTif in which all the question parameters including statis¬ 
tics are stored on disc, then comes into its own. Computeri¬ 
zation also facilitates book-keeping and enables the bank 
administrators to keep track of what they have got. It may 
be noted, however, that it will be necessary to retrieve 
the actual question material from the filing cabinets or wherever 
it is kept. It is quite feasible to store the text in a computer, 
costly though it might be; but pictorial material and certain 
mathematical and chemical symbols are beyond the reach of 
available technology;certainly on a routine basis. 

Questions/items ^tre also stored in the form of 
brochures and files, Th^se are particularly useful in the hands 
of teachers involved in continuous internal assessment making 
up periodical class tests, tutorials, assignments and so on. 

Also whenever a sample of questions/items of the banks main¬ 
tained by the university for purposes of end university exa¬ 
minations, is to be tried out for familiarity and practice 
by teachers and students* a booklet form will be quite use¬ 
ful, A Kardex tray with 4 compartments can easily hold 
around 5000 cards and the cost of this will work out to 
R8. 1500/- (Rs. 500 for tray; Rs. 500 for printing cards 
and Rs. 500 for typing). 



SECTION VI 


USING QUESTION BANKS 
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USING QUESTION BANKS 


People are often curious about how question banks 
work. Without going into the nuts and bolts, an attempt 
w£ 22 ese^aCiAl working as 

it affects the user. Firsts be wary of anyone who says 
he has a question bank. As often as not, it will turn out 
to be a scruffy collection of questions thrown into a filing 
cabinet, from which choosing questions is like finding 
bargains in a jumble sal^. 

Depending upon the form of the question bank, its 
use will be varied. In th^ simplest form of a file of 
typewritten questions/ite^ns with an individual teacher, the 
use may be limited to tl|e class he teaches for making 
up tests at the end of eve^y lesson or more practically at 
the end of a few lessons. The teacher then has to sort 
out the content matter for his test together with the out¬ 
comes or objectives he w^nts to test. He may then decide 
to have from each content area certain number of items 
checking recall (knowledge), certain other number of items 
checking comprehension (translation, interpretation and 
extrapolation), application etc. He will have to make a 
choice and constitute his test, administer it to his students 
and by conducting an iteni analysis for FV & DI, will be in 
a position to conclude about his items. It may also be that 
a brochure containing a certain number of questions/items 
will give enough details to the teacher-user to select 

cis.ss t€sts ill c^rt^iit select 

content topics for certain abilities needed to be examined. 
Sometimes the teachers niay write to he bank organizers 
giving their requirements and it is then for the bank 
organizerf; to make a choice and build up a test. This will 
enable a systematic feedback at the bank organization. 
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A blue print or a Table of specifications needs 
to be made in a university set up, ^en papersetters are 
invited to use the question bank question/items for paper 
setting. The Table will list the content topics as one 
dimension and the objectives or abilities like knoyrledge, 
understanding, application, analysis, synthesis and evaluation 
as a second dimension. Total weightages (relative) are to be 
allotted to each content area depending upon the answers to 
questions like 


1. To what extent an understanding of this area will 
help understanding other areas in the same 
subject? 

2. To what extent an understanding of this area 
will help understanding of other areas in other 
subjects in the course? 

3. What is the time spent in instruction in this 

area? 

4. Will the student use much of this area in everyday 

life? 


These total weightages in each area will be distributed to 
various abilities and a typical table will look Uke this 







Looking at the content area A, the breakdown is 4% for 
knowledge; 4% for understanding; 2% for application and 2% 
for skills. The papersetter has to siinpl 3 r choose questions/ 
items in this ‘content area A from the book . Perhaps he 
can take 2 objective type test items for 2 marks and 1 short 
answer £o^ i marks all checking knowledge; 1 short answer 
question for 4 marks involving testing comprehension; 
short answer question for 2 marks checking application and 
2 objective type items for 2 marks checking skills in computing 
etc. 


A further table as given below will help choice 
of questions/items 


Objectives 

Content 

Knowledge 
Obj SA LA 

Understanding Appdieation Skills 
Obj. SA lA Obj. SA LA ObjSA 

Total 

4 

A 

2 1 - 
(2 marks) 


1 - 

(4 marks) 

• 1 2 - 
(2 marks) 

12% 

B 

4 1 

(2 marks) 

4 

2 - 

(2 marks 
each) 

2 - 1 - - 
(6 marks) 

22% 

C 

3 1 

(2 marks) 

4 

1 

(3 marks) 

2 1-2 
(3 marks) 

18% 

D 

: 4 2 - 

( 2 marks) 

4 

2 - 

(4 marks) 

2 - 1 - - 
(8 maks) 

28% 

E 

2 1 - 
(2 marks) 

3 

2 - 
(3 markfl , 

2 marks ( 

1 - 1 

(6 marks) (2 marks' 


4 


5 6 


5 8 


6 2 3 


1 




u 

Number of objective 

teet itema » 40 

S.A* questions *17 

L.A. queetiona « 3 

Part A 40 itema 40 marka 40 minutea 

Part B 17 queatioaa 40 marka 95 minutea 

Part C 3 queationa ZO marka 4$ minutea* 

Part A ia made up of objective type teat itema and given 
fir at to atudenu. They may reapond on the queation book it- 
aelf or a aeparate reaponae aheet be prepared and given. 

After the completion of Part A, Part B and C wilt be given 
for anawering* A key to Part A and anawera to Part B and 
C muat be prepared before atarting correcting acripta. 

It ia inherent in the working of queation/ltem banka 
that those questions and items selected and put into a teat 
or examination^ muat be item analysed for FsclUty Value and 
Diacrimination Zndex» coasldaiiag the performance of 
students. A detailed procedure tor finding out FV and DI 
ia given in the Section VU . Pre and Post VaUdation of 
Queationa/Items. These values of KV it DI muat be entered 
in the cards carrying these queationa/items. 

»iao possible that teachers can use the banka 
of queations/itema for making up class tests much on the same 
lines as outlined above for university examinations. Perhaps 
a rigorous table of specifications may not be all that 
necessary for a class test at the end of a few lessons. It 
Is certainly r- ired when a class test is given at the end of 
a sufficient Cw,«rage of topics. An item analysia as above 
must be carried out and char a cteri Stic t entered on cards. 
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After sometimej questions/items (whose F. V, • & DIs 
known) ten be used out of a Question bank to create 
criterion referenced tests via* tests on which a minimum score 
of 75 io can be fixed. These are items of around that 
value of F* V. Tests and examinations can be constructed 
to diagnose weaknesses in learning. Suppose in a content 
area» the ability to apply oneself is required to be tested 
and diagnosed for its existence (or absence), then a series 
of application items in the area (of known FV and Dl) can be 
chosen and made up into a test. Those who return poor 
scores on this will be diagnosed to lack the ability to apply and 
the teacher can suggest certain remedial measures. The 
presence of a good collection of all kinds of questions/ 
items will enable different Boards of Studies and Examiners 
to increasingly think pf restructorlag the pattern of their 
papers. Teachers, examiners and others must imaginatively 
use the bank to serve different kinds of purposes always 
remembering the fact that item analysis is intrinsic to the 
operation and use of the banks. 



SECTION va 

PRE POST VAl,a:»^TION OP QUESTIONS/ITEMS 
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PRE AND POST VAUDATION OF QUESTIONS/ITEMS 
INTRODUCTION 

Tile core of Exemination Reform in Higher Education 
Uei in the reform of queetioni/iteme* Many of ub would 
agree on diia that if queationi/itemB are written by teachers, 
examiners and others with care in a scientific way, the main 
problem in reform of examinations is already solved. 
University teachers have been trained in die art and science 
of item writing byvarious agencies like UGC, AIACHE and 
ISTE but such trailing has neither systematic nor intensive, 
these teachers have also been trained to prepare blueprints 
for making their question papers, write questions/items to 
fit the Table of Specifications or Blueprint and thus build in 
validity. They have also been prepared to use a variety of 
selection (objective type) and Supply type questions/items. 

Many of these (few) trained teachers have not been utilised 
to prepare university examinations so much so the expertise 
they had acquired served them only to test their class 
students. Some of diese trained tqaohers, devoted and 
committed professionally tOj their work, assumed leadership 
role and trained other teachers similarly. Enough has not 
yet been done to have a real multiplier effect. 

r 

A few universities have attempted to train their paper 
setters/exaxxuners in scientific and rational methods of question 
paper making, judging the suitability or otherwise of dieir 
questions/items. etc». There are nnany university teachers/ 
examiners who out of their innovativeness ar? able to create 
good questions/item* but most of them are not aware of 
systematic prevalidation procedures to make judgement about 
the quality of questions/items. Boards of paper setters vdien 
they want to pass judgement about a particular paper set by 
an individual, paper setter, find it difficult to pass judgement. 
Teachers and o^ers have to be aware of criteria to judge 
the quality of different kinds of questions/items prepared by 
diem. This is particularly useful when a college or a 
university is in the process of constituting initial question 
banks or pooff in various subjects with the help of past 
examinations, item writers; teachers' and participants on 
specially conducted Question Banking Workshops. 
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The Celt of the AlU launched on a Question 

Banking Development Project in 10 first degree level 
BubjecU of Mathemaacs, Physics, Chemistry, Botany, 
Zoology, History, Geography, Psychology, Economics 
and Commerce, collecting questions/items from three 
distinct sources of : 

a) past examination papers of various universities 

b) item writers/teachers specially those trained 
in the area of examinations. 

c) workshops specially conducted for building 
question banks. 

In order to select the questions/items to be retained in 
the question bank, separate groups for each subject have 
been constituted. Each group consists normally of two 
subjects experts and an evaluation expert. Together they 
do 'prevalidation*. 

PREVALIDATION 

Prevalidation relates to the activity undertaken by 
each group towards passing a judgement about die quality 
of questions/items. In a universi^ set up. Boards of 
Examiners are appointed in various subjects to review 
individual question papers prepared by paper setters 
and such Examiners must be knowledgeable about 
criteria for judging the quality of questions in respect 
of validity, reliability and objectivity and so on. Ve^ 
often the only criteria, seems to be the closeness of 
agreement of the question paper with the previous ones 
and very often innovative departures are not encouraged. 

It is obvious therefore that such Boardsof Examiners 
must evolve a criteria to judge the quality of questions/ 

I terns and use them. 

The University of Calicut, in its pioneering attempt 
to constitute question banks in various pre and first degree 
subjects, tried to help their own teachers with a checklist 
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of criteria for ^|nr«v:ali<^li«»k of ^Boatioi»/it«&t for queation 
banks and a cdieckliat was proparad and ita oae 
illustrated during a Queatlim B^dctpg Vo^altop for die 
3.year degree course subjects* 

CHEKUST OF CRITERIA FOR PREVAUDAT lON OF 
QUESTIONS/ITEMS FOR QTOSTION BANKS^ 

CRITERIA TO JUDGE THE QUALITY OF SELECTI ON 

type ITEI^ 

A. GENERAL 

1.0 Is the item measuring an important learning 
outcome ? 

2.0 Is the item measuring an important coiitent 

2.0 area? 

3,0 Is the level of\di£ficuity likely to be right? 

4.0 Is die item Likely to be answered correctly 
by higher ability students? 

5. i? Zs cJre f^or liheiy de wrongly- 

by lower ability students? 

6.0 Is the item independent or does it overlap 
widi other items? 

B. SPECIFIC 

(i) Constant Alternative type 

1.0 Does the item include only one significant 
idea in each statement 

2. 0 Is the statement so precise that it can be 
judged unequivocally true or false? 

3.0 Is the statement short and In simple 
language ? 

4, 0 Poes the item use negative statements 
sparingly and avoid double negatives? 
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(u) Maitipla choice/MultipU i4c»t typga 

UO !• tba ttem conciie and unambiguous? Is 
the negative (if unavoidable). em|diasiBed? 

2*0 Is the stem a complete question by Itself? 
Does the item require the student to read 
die options to discover' what is being aiked? 

3.0 Is the content of the question clear? 

4« 0 Does the stem include anything that needs 

to be repeated in every option^ within itself? 

5»0 Are the options parallel in content? 

6.0 Are the options parallel in structure? 

7. 0 Is the item devoid of any clues such as 
mix up of singular, plural, precision and 
length of key option etc* 

8* 0 Is the key option unarguably correct? 

9.0 Are the distractors plausible? 

10. 0 Does the item exclude "alt of these" ? 

11.0 Is the language used in the item appropriate 
to the vocabulary of students at this level? 

12.0 Does the item avoid similarity of wording 
in both stem and the correct answer? 

13.0 Does the item exclude responses that are 
"all inclusive"? 

14.0 Does the item use an efficient format? 

(iii) Matching 

1.0 Does the item include only homogenous 
material in the "premises"? 

2.0 Is the number of responses sufficiently 
large so that the last of their premises 
can still have many options, to choose from? 

3.0 Does the item specify the basis of matching, 
type of matching, kind of entry etc. ? 

CRITERIA TO JUDGE THE QUALITY OR SUPPLY TYPE 

TEST ITEMS 

A. GENERAL 

1*0 Is the item measuring an Important learning 
outcome ? 
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2*0 la ihe Item meeaurlag an in^rtaiit content 

area? 

3.5 la die level of difflcultjf likely to be ri^t? 
4.0 la die Item likely to be answered correctly 
by higber ability atudenta? 

S. 0 la die item likely to be answered wrongly by 
lower ability students? 

6.0 Is the item independent or does it overlap widi 
odier items? 

B. SPECinc 


(i) Simple Questjon/completioB 

1.0 la the item so stated diet a single* brief 
answer is possible? (a word* a number or 
a phrase) 

2.0 Is the question direct? 

3. 0 Is the answer reUted to the main point in 
the statement? 

4.0 Does the item with die blank spaces make 
enoui^ sense so that a student knows what 
to do? 

S«0 Does the item in the case of a numerical 
answer indicate the degree of precision? 

(ii) Short answer Questions 

1.0 Is the statement of question simple* clear, 
unambiguous ? 

2.0 Does the question involve observable* 
measureable behaviour? 

3.0 Is the scope of die answer limited? 

4.0 Is the direction given in the question, clear? 

5.0 Is the question a valid testing situation for 
the ablUty considered? 

6.0 Is the question likely to be interpreted In 

the same way. by teachers/students/examiners?' 

7.0 Is the answer to the question capable of 
being marked objecUvely? 

8.0 Is the question likely to have the right kind 

of difficulty value? 
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9.0 la the question Ukely to be enawered 

correctly by many Ki^er etudenta? 

10*0 Is the question Ukely to be answered^ 

wrongly by many lower abiUty students? 

11.0 Is the question capable of further restructuring? 

(iii) Long answer 

KO In answering this question^ in your opinion 
does the student need to organise his ideas^ 
choose the form of his answer in his own words? 

2.0 Does the situation presented in the question 
seem to be new to most of them? 

3. 0 Is it possible that students can produce 
memorised answer to diis question? 

4.0 Does answering this question involve sort of 
judgement on the part of students? 

5.0 Is the time limit reasonable? 

6.0 Is the length and. scope of the answer 
specified? 

7.0 Does it avoid usage of very open verbs? 

USING THE CHECKLIST 


The checklist includes certain general criteria in 
respect of any type of question/item \^ethsr of selection 
or supply type. The question/ltem must be measuring an 
important learning outcome, an important content area; 
must behaving the right level of difficulty; must be answered 
by many higher abiUty students and by a few lower abiUty 
students: must not overlap with any other question. 

^re specific criteria listed that are peculiar to 
the item type. For example if the item is a True/False 
(constant alternative typel. we may have to check with die 
4 criteria listed under constant alternative, namely 
(i) whedier the item includes only one significant idea in 
each statement; (ii) whether the statement is so precise 
that it can be judged unequivocally true or false; (iii> whether 
the statement is short and in simple language etc. Similarly 
criteria specific to different types of items/questions are 

given in the checklist. 
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It may be ’necesgary to modify an item/queation to 
•atiafy aoma of the criteria. It may happen that aome 
qneatUma/ltema may have to be completely rewritten; yet 
aome olhera may have to be rejected totally. Queationa/ 
Itema that have been aelected for retention will have to 
be coded and referenced in terma of : 

(i) A bibliographical code number to locate it 
for uae in future. 

(U) Subject/toplc being teated 
(111) Type of queation/item 

liv) Objective being tested (A special coding has 
been designed ranging from Al, A2, A3...... 

G2 for all intelectual abilities ranging from 
•imple recall of knowledge through comprehension, 
anpUcation leading to evaluation at die highest 
cognitive level. The group doing prevalidation 
must look at the question/item and decide as to 
vhat ia the hipest level of intellectual 
abilities that the student has to display In order 
to answer this question/item. If they decide that 
comprehension interpretation ia the highest level 
of intellectual ability displayed by the student 
in answering the queation/item, a coding C2 will 
be given to it. 

(v) Given below ia a system of categorization to be 
adopted for questions/items : 

(A) In terms of content 

A bibliographical coding (number for each 
subject has been prepared. This will form 
part of Reference No. (Code NO.). 

(B) In terms of abilities/outcomes/objectives 

The followliig is the classicication with 
letter symbols* This will form (he other 
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part o!c Reference No* Code No. )• 

A1 Knowledge of specifies, terms, specific 

facts 

AZ Knowledge of conventions, trends and 

sequences, classification and categories, 
criteria, methodology 

A3 Knowledge of universal , abstractions, 

principles and generalisations, dieorles and 
structures 

B Skills of sketching, drawing, computing, 

reading 

Cl Comprehension - Translation 

C2 Comprehension - Interpretation 

CB Comprehension - Extrapolation 

X 

D1 Application A using information in concrete 
situations 

DZ Application - problem solving (blosed as 

well as open ended) 

£1 Analysis of elements and relationships 

EZ Analysis of organisation 

ri Production of a set of abstract relation, 

a plan or of proposed eet of operations 

G1 Judgement in terms of internal evidence 

GZ Judgement in terms of external evidenc^ 

Checklist of action oriented verns for all 
these Categories 

A1 define, describe, identify, table, match, 

name, outline, reproduce, select, state,* 

,AZ list, recall, detect mistakes, give classi¬ 
fications, name trends, list sequences, 

A3 recognise 

B sketch. 
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B iketch, draw, compute, find, calculate, 

tead chart# uae chart, draw graph, use graph, 
use tablaa, show by calculation, select table 

Cl deHne in own words, convert, distinguish, 
compare, estimate, explain in own words, 
estend, give examples, infer, predict, rewrite, 

C2 interpret, manipulate, translate, verify, 

discrlnunate, classify, construct, extrapolate, 
estimate future consequences, prepare, 

C3 represent, restate, rephrase, change, reorder, 
rearrange 

D1 compute, demonstrate, modify, change, find, 

calculate, prepare, produce, solve, show, 

DZ use, establish, verify, apply to solve^a 
problem, construct chart, graph 

£1 break down, differentiate, discriminate, 

distinguiel^ illustrate, outline, point out, 
rela.te , select, separate, subdivide, recognise 
£2 find^ relationship, establish relationship,. >tate ^ 

unstated assumptions, recognize logical fallacies, 
deduce, contrast, compare, derive 

FI categorize, combine, compile, compose, create, 
devise, design, explain, generate, organise, 
plan, rearrange, reconstruct, relate, 

F2 reorganize, revise, rewrite, summarise formulate, 
generalise, write a theme, conclude, constitute, 
transmit, modify, document specify, derive 

Cl appraise, compare, conclude, decide, constrast, 
criticise, discriminate, justify, relate, support, 
draw inference, judge adequacy, predict* 

Time for answering a selection type item; each 
item is usually given a minute. Sometimes to make 
the total test of 100 items (Multiple choice), 80 to 
90 minutes, alone would be given to make it speeded 
test for all those taking the test. 
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(Vi) M»fl» to be allotted: 1 mark is to begiven lor 

•very objective item. Anything between 3 to 6 marks 
to be given for short answer questions» anything 
between 6 marks and 16 xnarks may be given for long 
eiuwer essay questions/problem solving questions. 

PREVALIDATEP SAMPLE OT QUESTIONS/ITEMS 

Some samples of pre validated questions/items for 
these areas at first degree level are given to make it clear. 


Item The tangent at the point {I, Z) on the curve 

Y-rX^-12X4 18 is 

(a) Passing through the origin 

(b) Parallel to the X-axis 

(c) Parallel to the Y-axis 

(d) Cutting equal intercapts in the 
X and 'Y -axis 

(i) BibUografhical Code No. is 
516.36 

(iil Subject Geometry Topic Tangents» 
Norma lb to curves 
(iii) Multiple Choice 

(Iv) Objective tested: application-problem 
solving DZ Time for answering : 1 mt, 
(v) Time for answering :1 minute 
<vi) Marks; 1 


Ref. No. 

Class 

Subject 

Topic 

Type of 
Question/item 

sib.36 

First 

degree 

Geometry 

Tangent 
k Normal 

Multiple 

Choice 


Objective tested 

Level of 
Ouestioa 

Timefor Marks 

answerina allettsd 


D2 

First degree 

1 min. 

1 




(2) Physics 


Item 


A metal owes its magnetism to an orderly 
arrangemept of relatively large units (called 'domains') 
each composed of thou.^ands of atoms. In a magnetized 
material, - these domains may be compared to tiny 
magnets all pointing in the same direction. 

For each statement use (a) if it is supported by the 

principle 

(b) if it is contradicted by 
the principle 

(c) if it is neither supported 
TioT contradicted by the 
principle . 

(1) A current of electricity flowing in a nearby wire 
cannot cause a ba'* of metal to become magnetic 

(2) A hot bar of iron is harder/co magnetize than a 
cold bar 

(3) The best conductors are also capable^ of the 
strongest magnetization 


Ref, No. Class Subject Topic Type of 

_ __Question/Item 


First degree Physics Magne- Multiple facet 

tism 


Objective tested Level of Question Time for Marks 
__ answering alloted 

D2 First degree 3 mins 3 




(3) Chemiatry 
Item 


'If hydrogen sulphide is bubbled through an acidiAed 
solution of potasaium dichromatej solution 
turns bright green with an yellow precipitate* 

True False Don't know 


□ □ □ 


Ref* No* Class 

Subject 

Topic Type of question/ 

item 

First 

Chemistry 

Analysis Constant 

decree 


alternative 


Objective tested 

Level of 

Time for 

Marks 


Question 

answering 

allotted 

C2 First degree 

1 mint. 

1 


(4) Botany 

Item 

Use the following key and answer the item below: 

Kgy A) Both statement and reason are true 

E) Statement is true but reason is false 

C) Statement is false but reason is correct 

D) Both reason and atatement are false * 

1. Capparidae is not regarded to be advanced over 
Rannales because it has got difinite number of 
floral parts arranged in a cyclic manner 

2 , The plants belonging to the family Apocyana case 
show anamalous type of secondary thickening because 
of the occarrsDce of spicular cells in ttie plant 
parts 



3, The family Nymphaceae la regarded aa primitive 
becattae of the equatic habit oi the plai^eta 


Ref. No- 

Class 

Subject 

Topic 

Type of question 

5S3.0 

Pirst 

Botany 

Taxonomy 

Multiple Pac et 


degree 





Obiective 

tested 

Time 

for answering Marks allotted 

Cl 

- 


3 mints. 

3 


(5) Zoology 
Item 


Check the most appropriate answer. In an experiment 
the sponge is seqeeesed through silk bolting cbth in 
appropriate medium and allowed to stand, Por v^at 
purpose is this experiment perfomed? 

To prove that 

a) the sponges are multicellular 

b) the cells are loosely arranged and cell 
relatloBa can be disrupted 

c) ^'he cells are so small that they can be 
squeezed dirough cloth 

d) the sponges are nonliving 


Ref, No- Class 

Subject 

Topic 

Type of question 

593.4 IB.Sc. 

Zoology 

Organi- 

Multiple choice 



zation 



Objective tested 

Time for 

answering 

Marks allotted 

Dl 

1 

mint. 

1 




< 6 ) 

j|0m Tte husic determloAot of ebangoB In the quantitjr 
supplied of a product La 


?£! 


A) Technologlcnl advance 

B) Prices of the rewurces used In the productloo 
of the product 

C) Price of the product 

D) Expectations cmc^minf the future price of the product 


338:*h213 First Economics Price- MiUlple Choice 

degree supply 

_____ 

Al 1 min. 1 mark 


^^mmejM 

Item The combination in which one management acquires 
the control over different successive stages In 
production (n one industry Is called 

A) Vertical 

B) Horizontal 

C) Circular 
C) Lateral 

ISef/NoT”Class Street _ 

658.5 First Commerce Management Multiple choice 

degT^e Qfprod^tlon_ 

"" Obj^tve tested '"J _ 


1 mla 


1 mark 


Item The purpose of theory in Psychology Is 




Ref. No. 


A) to guide experiments 
B| to integrate and predict 

C) to explain new events in a field 

D) to offer detailed explainatlon of definitions 

ClaiB Subiect Topic Type of qucBtlon/Item 


Subject Topic 


First P sycholftgy Gen^a I 

Degree Psychology 


Multiple Choice 


Objeenve toeled Time for emwerijig Marki allotted 

















POST VAXJPATION 


Post validation refers to a systematic analysis of 
items/questions for two Important characteristics, namely 
4tem facility' and 'item discrimination index'. There are 
very rigorous statistical procedures developed for analysing 
accurately various items/questions contained in a test. 

At die aame time diere are certain simple methods developed 
for analysing items/questions and these methods yield fairly 
accurate values which are tor all practical purposes adequate 
for a clasaroom tsacner and for a university Board. There 
are techniques developed both lor objective type items as well 
for subjective type questions* These are presented under 
item analysis here. 

ITEM ANALYSIS 


The most interesting, challenging and rewarding part 
of test analysis is die item analysis. It is through item 
analysis 

a) that a teacher learns most about himself as 
a teacher 

b) that a teacher learns what his student^have 
(o^ have not) learned 

c) diat a teacher knows his skill as an examiner 

d) that a teacher can improve hisown item writing 
skill very rapidly. 

It is die purpose of item analysis to tell us about each 
item in our test two things :- 

1) How easy or difficult is the item? How many 
students answered it right? 

2) Is the item measuring the same thing as the 
rest of the test? Does it discriminate between 
the good and weak students? 

The response sheets of students on an objective type 
tefct are all collected and marked. One can do edgr mar¬ 
king or one can prepare a matrix of performance in a tormat 
specially designed with item numbers and student Register 



Numbers with & scheme that if an item 13 answered right 
by a studentr then there will be an entry of I and if an 
item is answered wrong by another student there will be 
an entry of O, ThU is repeated for all items and for all 
students. 

A key witii answers to all items in prepared and with 
the help of this marking could have been done. Even a 
computer can be used. This is particv'larly advantageous when 
we deal with a great number of etuder^s on a test with a 
good number of items. 

Their marked answer response sheets are arrang^c, 

In the order of rrierit; count pf the total number ol students 
who took the test fit 27% of this is to be worked out fwhy 27%| 

There are good and valid statistical reasons tor this; If 
the blserial calculated f-rom the extremes of a distribucion^ 
the standard error of this blserial is at a minimum when 
the two tails contain 27% of the cases each (Guilford 1954, 
Harper 19h5).. 

FACILITY VALUE 


The facility value or facility index of an item is the 
percentage of students who answered the item right. If we 
call the upper and lower groups t» be fairlv indicativeof 
the total performance, then 


FV 


Total rights in upper and lower groups 
2 X no. in each group 


Simply, one has to count the students who got 1 for the 
item in the upper and lowei groups but together and work 
out the percentage of the total 


FV can range from O to 100% 


Number right in the upper group and divide by number of 
either group. In formula from E)I=(Facility of upper group)* 
(Facility of lower group) DI is expected as a decimal value. 

Its range is possible from -1,00 to + 1.00. 
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Meaning of D,I. 
a) at trial test stage 

-1,00 through 0.0 to + 0,20 No significant discrimi¬ 
nation; item not related to 
the test as a whole. Upper 
sample not doing better on 
the item than the lower 
sample;modify or reject 

+ 0,2 to + 1,00 Good positive discrimina¬ 

tion 


b) after trial test; In use; 
-1,00 to -0.25 
-0.25 to +0.25 

+0.25 to +1,00 


Use in teat construction 


Weaker students have a 
better grasp of the topic 
than better students. All 
students' knowledge of the 
topic is equal 

Too high a positive 
discrimination^ will mean 
that 'gap' between better 
students and poorer stu¬ 
dents may be too great. 
Remedial work for poorer 
students may be required. 


1* Items with known discrimination indices may be used 
to generate a test with a required spread of marks. 
Using items with low DIS will produce a small spread 
in marks; items with high DIS will produce a larger 
spread of marks (and therefore perhaps more failures 
and/or more distinctions^ 

2. Items with known discrimination indices may be used 
to produce a test which has weak discrimination on 

earlier items, the discrimination in favour of better 
students increasing through die test. 
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Discrimination indices may be used to lo.cate those 
topics whi^ all students will learn; and ttiose topics 
which favour those students who otherwise are not 
doing well. 

Tables I and II show the performance matrix of the upper 
ability and lower ability groups of students (27% of 44 
students s 12 students) in both a 20 item objective type 
test as well as a test of 10 short answer questions. 

Facility values of each of these 20 items will be 
calculated by finding the total for every item of all the 
upper ability 12 students and the total for every item of 
all the Lower ability 12 students and dividing the sum of 
the two totals by 24, For example item (1) has 12 from 
upper ability and 11 from lower ability 12 thus a total of 
23 out of 24 


F. V, = ^ X 100 = 95. 83% 
Z 4 

12 


F. V. of upper ability group is x 100 = 100% 

F.V. of lower ability group is x 100 = 91*67% 

F.V, of the total = -L0P.+.91,67-^ 95,83 

2 


D,I.of the same item is - ^ 2 ) " ^ " 0,082 

Facility values of each of these 10 questions of 5 marks each 
will be calculated as under. 


Find the total marks obtained by all the upper ability 

12 on any question and divide this total by the maximum 

possible total for diat question by all 12 students. This 

will give the facility of fee upper group. For fee first 

question this is ^ = 100%. Let us do this wife fee 12 lower 

ability. This is 100 = 50%. F.V, of the ^ole 

60 


group 


100+50 


75% D.l. 


100-50 = 0,50. 


2 


100 



StandArds preAcrib^d for good item*/q«« 0 don*• 

F*V. for both objective and Abort anAwer, long anAwer etc. 
muAt range from 20% to 80%. 

D.l. for objective type items 0.2 to 0.6 

0.6 Excellent item 
0 to 0.2 needs improvement 
0 to be discarded 

D.l. for short answer/long answer 0.5 (acceptable) 

0.3 to 0. 5 needs improve¬ 
ment 

0* 3 to be discarded 

Meaning of F. V. 
a) At trial test stage 

0,0 to 25% item too hard* modify (check distractor) 

25% to 75% item of correct facility. 

75% to 100% item too easy reward or reject (check 

for clues) 


b) after trial teat? i. e., in use; 

0 to 25% topic not taught well/not learnt well 
(check teaching techniques) 

25 to 75% topic reason ably well taught/leanrt well 
75 to 100% exceptionally good knowledge of topic. 

Use in test construction; 

1# Items with known Facility values can be used to design a 
test widi a predictable difficulty. 

2. Items with known Facility Values can be used to design a 
test which starts with "easy" items and progresses to 
"harder*' items. 
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3. Facility Vaot^a give a quantitative measure! of ^e 
learidag 9ie topic and ability tested by the item. 

4 ^ ncLUty Values allow different groups of students to 
be compared, topic by topic on a subject Ihey^ are 
all studying (eg. College performance with National 
or Aegioaal performance or this year's students 
with last year's students). 

5. Patting teats together from a pool of Items with 
known Facility valoos enables "Standards" to be 
defined and maintained between courses or over the 
years for a given course. Recommended 
facility = 20% to 80% (25% and 85% Paul Dlodorlcb) 
Average facility of all items = 50%^ 

Precision of measurement isgreatest when aU of the items 
in a test are about equally difficult for the group tested, 
tha maximum reliability and dispersion of scores could 
be attained if every item (M-C test item) is answered 
correctly by somewhere between 60% to 70% of students 
(Paul Diederlch I960). Ebel has shown in hisbooktp. 363) that 
the wider the distribution of item facility the narrower will 
be distribution of test scores. The ablest students sre 
those who can answer the largest number of| items of 
average difficulty (Harper). 

DISCRIMINATION INDEX 


How well the item is able to discriminate between 
higher ability students? Even though there are 60 different 

4 

item analysis methods (Males, 1972), it turns out diat 
one of the simplest is also one of the best for oriinary 
use. This is Johnson's upper-'lower index (Jc^nson 1951). 
Its calculation is really simplicity itself. Ifou substract 
die number right in the lower group from die number 
right of the upper group 


SJ 


DISCRIMINATION 
INDEX (Dl) 

FACILITY VALUE % 

Below 40% 

40 to 60% 

Above 60% 

Above 0.40 

Difficult^ 

Accpetable ^ 

Easy’S* 

0.30 to 0.39 

OifficuU 

Improvable 

Easy 

0.20 to 0.20 

Difficult 

Marginal 

Easy 

0.20 

Rejected 

Rejected 

Rejected 


* Acceptable!* for a test 


In general very easy and very difficult Items have 
poor dlscrimination and too many of these would have a 
serious effect on the reliability and validity of the test. 

Further points 


1. Facility valuesand discrimination Indices will have 
most meaning at the pretest stage if the pretest 
form is administered to a sample of a respectable 
number of average (or typical) students. 

Z. The two basic item statistics should always be 
recorded with the item along with details of the 
pretest sample if item banking (i.e. of items) 

LB to be undertaken. 

3. Computer programmes are increasingly available so 
that this anaiysiB could be in a few minutes. In 
addition to the programme, the computer will require 
the student selections (a, c, c, d etc. \ either copied on 
to punched tape, punchedcarda or a teletype or 
directly read by a mark sensing device. 

Effectiveness of Distractor 

Illustration ; 

1. Choice A C D 

Upper IS 1 10 3 4 

Lower 18 14 6 7 

Key has positive.* Discrimination 
distractors are D ^ need5 to be looked at. 



2. Choice A B* C O* 

Upp«r 18 7 2 1 8 

Lower 18 2 7 4 5 

Keyed answer has positive discrimination 
B Ik C Distractor are fnnctloninf well. 

A distractor has positive llscrimination 
(A sottst be revised) 

3« Choice A B CD 

Upper 18 3 4 6 5 

Lower 18 2 6 3 7 

Key has negative discrimination. 

A Ik C Distractors have positive 


discrimination (They have to be 

revised) D is 

O.K. 


Choice A 

B 

C D"-^ 

Upper 18 2 

3 

10 3 

Lower 18 4 

5 

3 6 

Key has negative 

di s c r imlnation. 


C Distractor has positive discrimination 
A It B Distractors are alrifl^t. 

Wording of C It D will have to be looked 
into. May be D is wrongly keyed. 


Choice 

A 

B 

C 

D 

Upper 18 

16 

1 

1 

0 

Lower 18 

15 

2 

1 

0 


Item too easy to discrlmLnate 


Summary 

Ihere are very many sophisticated "sample free" 
techniques for finding FV It D1 that have been developed 
(Wiiimott Research in Education^ May 1975). 



Since a relatively tmall number of studenU ia 
Involved In claesroom testit item analysis results are 
to be interpreted widi great caution. FV and Ol calculated 
on the basis given above will be subject to sampling 
instability i. e. tliey will vary from one group to another. 
We can however favour Items at 50% FVor around and 
those with the highest discrimination index. 

If an item; 

a) Provides positive discrimination (0.20 and above) 
alternatives function very well 
c) measures a significant objective 

it should be retained or placedin a file or Bank. It is 
better to analyse the item everytime it If used and study. 
This will clearly show if the item has been more or Less 
consLstant or showing variability in indices. 

Those who are interested in knowing more about 
diis, are invited to correspond with the Project Officer 
(Examinations), AIU who will be extremely glad to help. 

FOLLOW UP St FUTURE V/ORK 


It is hoped that teachers, examiners and others, 
initiated into the science of item writing/question making 
and prevaUdation of these questions will in future be 
engagedin similar activities of item writing, prcvalidation 
and post validation. Each one of them is a professional 
activity, associated withteaching. At the same time, these 
personscan assume 'eadersKip functions and train a good 
nvimher of colleagues in their own institution, thus producing 
a multiplfefeffect. This is perhaps tlic only way in our 
country to train a great number of teachers and examiners 
in a short time with less financial commitment. 

CONCLUSIONS 


It is vital that all teachers engaged in university 
teaching are trained in the science of item writing/question 
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mateUig. A c^eme by nbldi 4 gTO«^ of Loaders cmxk bo 
tboroagbly traliiod aadtpidi the bolp of secondary traUdag 
pn^rammsa by tbere Leaders in &elr colieges/ 
vnlirertitles/institutions^ is proposed on the basis of 
experience gained in working out such a scheme with 
stnne universities in the country* It is also equally 
important that teachers, examiners and other must bo 
trained in prevalidation and post validation of questions/ 
Items so that they will be able to make useful and 
meaningful inferences about 'their own questions/items* 
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SECTION 




FUTURE OF QUESTION BANKS 



FtJTURS or OITESTIOH BAI^KS 


Baoictiii^ rcsnaias ia fli« categaxff o/ 
proanieisg idea*. A law devalopmetits could ehaaga tliia 
aifctatioa* Umversitfcaa/caBaiaiiriTtg boards could Joto 
togodior, pool rosourcoa so diat t^jC^rs could bagiu 
to reqt^atf more voeHerouaip Ibaa in tfeo past, acesra, 
to bank of quesUons/lbsms for^ tlla purpose of classroom 
testiitg Bmd costiaiioiis sosesament^ S<une universities 
have already built up question bssiks udiich they could 
put at the disposal of teachers* The AlU has developed 
nearly 60,000 questions/items in 10 subjects (Mathematics, 
Chemistry, Physics, Botasiy, Zoology, History, Psydiology, 
Geography, Economics ami Commerce) at the first degree 
level and can make them available either in the form of 
cards in Kardnc trays as a system or In the form of 
booklets In A4 sise double column single space arrange* 
ment. At the same time nothing must stop local groups 
of teachers banding to|^other immediately to form their 
own question banks using their own and odier materials* 

It is also proposed to take up other tpojmlar subjects 
at the First degree level in future* It is firmly 
believed that item/question banking is the examining 
system of future* It is also felt that a few large 
universities can try this out and make dieir experiences 
available to all others. 

The question of die cost of LtMn question banking 
In comparison^with other systems of examining and 
achieving comparability isan important one. At an 
initial level, th€ rationalizing effect of question 
banking would seem bound to make it the cheapest 
examining method of all* Yet we are unable to 
demonstrate these advantages simply because no 
comparison of the cost effectiveness of the various 
examining and comparability procedures has been worked 
out. Some idea of the relative costs can be obtained by 
totalling what it currently costa to examine candidates in 
say these ten subjects every year* To this can be added 




the Boe^de aad/er ottLverei^e pef either to 
heve SB effective croee moderetlng aetwork or to coBtlaoe 
with eaau&l monltoriag ezereleee. It will be eor^rleiai If quee* 
tloB benke in these eubjecte cannot be operated for conalden^ly 
lees than thie amount. 

Plane of action for setting up question banks In 
individual universities have been given elseehere. Some 
universities have launched upon this; others are on their 
way to. The function of toe AlU is to coorditiate toese 
efforts and also institute initial fairly large queetioas/items 
banks in more popular Bret degree level subjects and make 
them available to all universities in the country* However^ 
toe totttra of question banks in tois country at toe level of 
higher education depends to a very large ac.tent on toe 
awareness and understanding of a system of queetioB bank 
on toe part of administrators« Boards of Studies members, 
ejuuniaers and teadier«» There is no doubt that toie will be 
successful over a period of time since toere is sample 
evidence to showtoat individual universities/colleges/other 
organieatlons a^re keen and entousiastic to develop 
question bank in many subjects. Thus question bank will 
make a potential contribution and add a new dimension to 
toe proceee of cutamining In higher education. 
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